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MACHINE PROCESSING OF S-192 AND SUPPORTING AIRCRAFT DATA ~ 

STUDIES OF ATMOSPHERIC EFFECTS, AGRICULTURAL CLASSIFICATION, 

AND LAND RESOURCE MAPPING 

1 

INTRODUCTION 

This is the final report on contract NAS9-13272 dealing with a 
study of changes in pattern recognition performance caused by variations 
in atmospheric state over the area of interest. As a secondary goal, 
we assessed the utility of S-192 data for mapping terrain types in the 
Cripple Creek, Colorado area. The work was conducted in the Infrared 
and Optics Division by Frank Sadowski and Thomas Austin, under the 
direction of Fred Thomson. The NASA technical monitor was 
Larry B. York. 

1,1 STATEMENT OF THE PROBLEM 

To be cost-effective, earth resources surveys using satellite or 
aircraft multispectral data must be done over large areas. If pattern 
recognition techniques are used as an information extraction tool to 
provide the user the information he wants from the multispectral data, 
those techniques too must be effective over large areas. 

The use of pattern recognition techniques involves the assumption 
that the spectral radiance signatures of objects or classes to be 
recognized are sufficiently unique that these classes can be separated 
by the pattern classifier or recognizer. Further, training sets are 
required, either to teach the recognizer what the classes to be 
recognized look like or to evaluate which spectral clusters correspond 
to which objects to be recognized. Because time taken to train the 
recognizer cannot be spent processing data, we would prefer to train 
as infrequently as possible. 

In operations, we must retrain whenever the conditions of objects 
change materially (e.g., vegetation phenologic state) or when the 
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conditions of measurement change (e.g. solar illumination, atmospheric 
state, or sensor performance). Preprocessing techniques can partially 
compensate for changes in measurement conditions and allow an extension 
of the area over which a given set of training set signatures will 
yield good results. 

But utilization of preprocessing techniques may also slow the 
processing of data, especially when parameters for the preprocessing 
algorithms must be estimated from the data. Hence the question 
arises — How often, or under what conditions does pattern recognition 
performance suffer so much that preprocessing must be performed (or 
existing preprocessing al- ered) to retain good accuracy? That is the 
question addressed by the first phase of this vjork. 

The second phase of this work addresses the utilxty of S-192 to 
map terrain features in the Cripple Creek, Colorado area. The ability 
to map these terrain features accurately and periodically is of value 
to Federal, State, and Regional resource managers in their managt-.iient 
of the area. Since S-192 provided a wider selection of spectral bands 
than did ERTS , a comparison of the utility of S-192 and of ERTS data 
would reveal the improvement in performance through use of more nearly 
optimum spectral bands. 

1.2 SUMMARY OF APPROACH 

The approach to assessment of changes in pattern recognition 
performance caused by variations in atmospheric state was to simulate 
changes in performance caused by variations in atmospheric optical 
depth and in terrain base elevation. Then the realism of the simulation 
was checked by classifying S-192 data collected under known atmospheric 
conditions and comparing results with those of the model. Three cases 
were simulated with the model. In the first case, the effects of 
varying atmospheric visibility at 2,000 ft elevation were assessed 
In the second and third cases, the effects of varying base elevation 
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under particular atmospheric states was assessed. A clear atmosphere — 
high elevation case (40km visibility and 6^000 - 14,000 ft elevation 
range) and a hazy atmosphere — low elevation case (10km visibility and 
2,000 - 6,000 ft elevation) were simulated. 

Basically, reflectance signatures of common midwestern crops were 
calcul.:i-,ed from aircraft multispectral data using procedures further 
described in section 2.3. Then the different atmospheric states and 
base elevations were simulated, and path radiance, path transmission, 
and solar and diffuse irradiances calculated using a model developed by 
Robert Turner [1] . The reflectance signatures were converted to radiance 
signatures at each of the atmospheric states. A computer program 
simulating the ERIM linear classifier was written. It was trained on 
radiance signature sets collected under base atmospheric conditions, 
then it classified samples taken from the radiance signatures generated 
for other atmospheric conditions. In this way confusion matrices of 
performance were generated for several different cases. 

The approach to the Cripple Creek study was to prepare a recognition 
map of terrain features using S-192 data and training sets selected by 
Dr. Harry Smedes of the U. S. Geological Survey, Denver, Colorado and 
Jon Ranson, a Colorado State University graduate student assisting 
Dr. Smedes. From an analysis of training set signature statistics, 
seven optimum S-192 bands were selected for final mapping. The final 
map accuracy was assessed and results were compared with a similar effort 
using ERTS-1 data [10] ■ 

1.3 SUMMARY OF CONCLUSIONS AND RECOMMENDATIONS 

A simulation study of the effects of atmosphere on the performance 
of pattern recognition devices was performed. Also, computer assisted 
land resource mapping was performed on an agricultural scene in Michigan 
and a natural vegetation scene near Cripple Creek, Colorado S-192 and 
supporting aircraft data were used for the Michigan study, while only 
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S-192 data were used for the Colorado study. As a result of these 
studies, the following conclusions were made: 


1) Simulation studies provided an accurate measure of 
the effects of changes in atmospheric visibility 
and of base elevation on the classification of 
agricultural crops. This technique could provide 
additional useful information if used with atmospheric 
models of proven validity (such as Turner ^s) with 
accurate levels of sensor noise, and with realistic 
crop or other signature reflectance statistics derived 
from low and medium altitude aircraft data. 

2) In simulation tests of crop recognition accuracy using 
simulated ERTS bands (which were narrower than the 
actual ERTS bands) and four "optimum" S-192 bands, 
with accurate simulations of ERTS and S-192 sensor 
noise, the four narrowed ERTS bands provided nearly 
the same performance under training conditions of 
atmospheric visibility and provided 65% training 

set recognition accuracy or better over a wider 
range of visibilities than the four "optimum" bands. 

3) Seven optimum channel simulated crop classification 
performance was marginally better than the four optimum 
band performance with S-192 noise conditions at the 
training condition, and performance was more extendable 
over visibility variations than the four channel results. 

4) Under conditions of no sensor noise — with variations 
in signals caused only by terrain variations — the crop 
classification results were extendable over larger ranges 
of visibility and base altitude with the narrowed 

ERTS bands than with the four or seven optimum bands. 

This occurred because the ERTS bands were located in a 
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ispectral region where atmospheric effects were 
reasonably small, while both the seven and four 
optimum bands included bands in the blue-green 
portion of the spectrum. In using data from 
newer spacecraft sensors (with generally higher 
signal to noise ratios than either ERTS or S-192) 
for agriculture surveys one should consider band 
selection (both number and placement) in the light 
of the extension of signatures over a range of 
atmospheric conditions rather than solely on the 
performance in separating training set signatures. 
The ability to preprocess data is also an Important 
consideration, and the results presented shed some 
light on how well preprocessing must be performed. 

5) Using S-192 data collected in August 1973 over the 
Michigan Agricultural Test Site near Lansing, corn, 
woods, bare soil, and soybeans were mapped with 
71.1, 81.4, 85.7, and 50.0% accuracy respectively. 
The low performance on soybeans is attributed to 
sparse crop cover as a result of extremely late 
planting in the 1973 growing season. 

6) In Cripple Creek, Colorado land resource mapping, 
six optimum bands were chosen from the thirteen 
available S-192 bands. The bands selected were 
narrowed versions of the four ERTS bands plus two 
bands in the 1.5 - 2.35 pm region of the spectrum. 
The thermal band was not selected even though the 
data were collected near midday, probably because 
of the relatively high 2.3°K noise equivalent 
temperature (NET) in the thermal band. 


13 



FORMERLY WILLOW RUN LABORATORIES. THE UNIVEPSFTV OF MICHIGAN 


2 

ASSESSMENT OF ATMOSPHERIC EFFECTS ON PATTERN 
RECOGNITION PERFORMANCE 

This section contains a discussion of the portion of this project 
dealing with the assessment of effects of variation in atmospheric 
optical depth and of variation in terrain elevation on the performance 
of a pattern recognition algorithm. The basic rationa.le for such a 
study has been presented in section 1.1. This section focuses on 
procedures! analysis of results, and discussion of implications of 
those results. 

2.1 APPROACH 

The approach to this effort has already been summarized in section 
1.2. In this section, the approach is documented in more detail. 

Figure 1 schematically details the approach. 

First, we generated reflectance signatures of various midwestern 
crops to use in further simulation studies. These were generated from 
aircraft multispectral scanner data collected at the same time as the 
Skylab SL-3 overflight of the Michigan test site on 5 August 1973. 

Data from the ERIM M-7 scanner were smoothed to simula+'e 80m resolution, 
and in the process, reduce digitized sensor noise to negligible 
proportions. Then the data were caliorated to reflectance by use of 
reflectance panel and secondary reflectance standards. Finally 
reflectance signatures for the common crops in the scene were generated. 
See Section 2.2. Wavelengths of the M7 scanner channels are given in [6], 

The atmospheric model of Robert Turner was used to generate path 
radiance and transmission and total irradiance for a variety of 
atmospheric visibility and base elevation conditions. Calculations 
were made for the centers of the ERIM M-7 scanner bands. A 2,000 ft 
elevation study of variations of atmospheric parameters with variations 
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in visibility was performed. Also, two base elevation variation studies 
were performed. For 10km visibility the base elevation was varied from 
2,000 - 6,000 ft while for 40km visibility the base elevation was varied 
from 6,000 - 14,000 ft. See Section 2,3. 

After calculating atmospheric parameters for a variety of atmospheric 
and base elevation conditions, these data were used to calculate crop 
radiance signatures for the crops whose reflectance signatures had been 
previously calculated. The radiance signatures, simulations of what a 
sensor would see under these atmospheric conditions, were used in a 
model simulating the pattern recognition device. We used the model to 
calculate the performance of the pattern recognition device, assuming 
it had been trained on signatures collected under one atmospheric 
condition. Confusion matrices were generated showing performance as a 
function of atmospheric state and base elevation. Since these results 
were generated using d'-ita generated from training sets only, we refer 
to these results as "training set" results. To simulate effects of 
sensor noise on classification, two simulations were run. The first 
was run on noise— free data and the second was run on data in which the 
effects of sensor noise had been included. See Section 2.4. 

S-192 data from the Michigan Test Site area were classified, using 
nearly the same categories as for the simulation study. Atmospheric 
measurements made at the time of Skylab overpass were used to determine 
the atmospheric state. Results of evaluating the Skylab map accuracy 
were compared with model results to determine model validity. See 
Section 2,6. 

Last, analysis of the results was done to assess the range of 
variation in base elevation and atmospheric state over which good 
pattern recognition performance could be retained. See Section 2,5. 

2.2 AIRCRAFT DATA PROCESSING 

The purpose of processing aircraft data collected in support of 
the Skylab overpass was to derive directional reflectance signatures 
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for use in the model described in Section 2.4. For this purpose, data 
collected at 1,000 and 10,000 ft over the Michigan Test Site were used. 
The data were collected on 5 August 1973 at 0830-1130 EDT. 

The starting point for analysis was 10,000 ft data already digitized 
and converted to 7094 computer compatible format and 1,000 ft data 
digitized and scaled but not yet converted to ERIM 7094 format. See 
Figure 2 for a flow of aircraft data processing operations. 

2.2.1 1,000 FT DATA PROCESSING 

Data were first converted to 7094 format to facilitate further 
processing. Then, using gra3nnaps already generated for CDC 1604 computer 
processing steps (on another project), the locations of each of the 
gray reflectance panels and of the secondary standards were found. 

Signatures for each of the panels and for the standard fields were 
extracted using the STAT program. The next step was a calibration of 
each channel of data in terms of reflectance, using panels signatures 
and panel reflectance values from report 101700-10-X [3). As shown in 
Figure 3 the panel means (in integers) were plotted versus the panel 
reflectances and a best fit regression line determined for the data. 

Two constants, an additive constant a and a multiplicative constant b 
were determined such that: 

reflectance = a + b (integer value) 

These constants were determined so ?s to minimize the mean square error 
of the fit of the line relating panel signals to reflectance. 

Once ’’the constants a and b were chosen for each spectral channel, 
the reflectances of each of the secondary standard fields (used to 
calibrate 10,000 ft data) were determined through an application of 
equat ion 1 . 

2.2.2 10,000 FT DATA PROCESSING 

The processing of the 10,000 ft data began with a smoothing of the 
basic data to 80m resolution. The data were digitized with 14.4m 

17 




M 

00 



FIGURE 2. FLOW OF AIRCRAFT DATA PROCESSING 


F^RMEI^LY willow run 4.ABORATORIE5. THE UNIVERSITr OF MICHIGAN 












Panel Mean Signal (integers) 


2pi 


FORHERLy WI1.LOW RUN LAEaRATORIES. THE UNIVERSITY OF MICHIGAN 



Panel Reflectance (%) 


FIGURE 3. TYPICAL PLOT OF PANEL SIGNALS VERSUS 
REFLECTANCE (line shown is best fit) 
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resolution along the scan line and 12.19m resolution along the flight 
line. To smooth to a square 80ra element required a smoothing operation 
of 6 X 7 across the scan lj.ne and along the flight line respectively. 

After the data were smoothed to about 80m (the actual figures are 
86.4 X 85,3m), the same secondary standard fields located on the 1,000 ft 
data were found on a 1 x 1 graymap of a red band and the signatures 
obtained. Then a calibration of the data was performed by determining 
constants a and b relating digital counts to reflectance in exactly 
the same way as was done for the reflectance panel data in the 1,000 ft 
data set. Separate a and b values were obtained for each spectral 
channel . 

The next step was the extraction of signatures from large fields 
for the crop types bare soil, trees, corn, soybeans, oats, pasture, 
stubble, dense green, and water. Large field data were used so that 
the same fields can be located on Sky lab data. Multiple samples of 
each category of crop were selected and signatures extracted from the 
combined area. 

During the combining step, the results were scaled to reflectance 
by Imposing the scaling coefficients (the b values previously determined) 
in the combining operation. Later revised mean value cards (representing 
the difference between the scaled mean and the constant a) were punched 
and inserted in the signature deck. 

The ref.ults of the aircraft data processing, the reflectance 
signatures, are shown in Table 1. Shown there are mean values and 
standard deviations for the nine crop types previously mentioned. In 
interpreting these results, one should remember that these signatures 
represent the appearance of crops at 80m resolution. Also, at the 
extreme values of smoothing used on the original aircraft data, the 
standard deviations shown represent the variation in the reflectance 
of the training sets; the sensor noise also present on the analog tape 
recorded data was reduced to negligible proportions by the extensive 
smoothing of the data. 
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TABLE 1 


REFLECTATrCE MEAltS AliD STANDARD DEVIATIONS 
FOR MICHIGAN AGRICULTURE CROPS “ 5 AUGUST 1973 

(scandard deviations in parentheses) 




.A1 " .48 

.46 - .49 

.48 - .52 

.50 - .54 

.52 - .57 

.55 - .60 

.58 - .64 

.62 - .70 

.67 - ,94 

1.0 - 1,4 

1,5 - 1.8 

Class 

I 

2 

3 

4 

5 

6 

7 

e 

9 

10 

11 

Dense Corn 

2,65 

3.07 

2.91 

4.17 

5.62 

5.31 

4.39 

3.45 

30.47 

37.02 

14.54 


(.276) 

(.264) 

(.237) 

(.297) 

(.502) 

(.471) 

(.313) 

(.225) 

(3.36) 

(2.85) 

(1.18) 

(foods 

1.68 

1.65 

1.52 

2.37 

3.55 

3.02 

2.20 

1,56 

25.06 

33.24 

11,51 


(.737) 

(.674) 

(.575) 

(.614) 

(.739) 

(.804) 

(.757) 

(.609) 

(.368) 

(.416) 

(.244) 

Bare Soil 

10.41 

11.30 

10.59 

12.27 

13.20 

15.63 

15.74 

15.63 

20.05 

29.65 

33.82 


(3.58) 

(3.97) 

(3.60) 

(3.58) 

(3.75) 

(4.34) 

(4.52) 

(4.29) 

(7,78) 

(9.44) 

(6.90) 

Sparse Corn 

3.63 

3.96 

3.73 

5.51 

7.58 

7,29 

5.80 

4.53 

29.12 

36.39 

18.90 

(.753) 

(.902) 

(.765) 

(.755) 

(.891) 

(1,04) 

(1.19) 

(1.12) 

(3.36) 

(4.50) 

(3.00) 

Stubble 

4.45 

5.44 

5.22 

7.17 

9.26 

10.00 

9.76 

8.81 

24.50 

37.60 

30.96 


(.429) 

(.572) 

(.592) 

(.607) 

(.544) 

(.793) 

(1.18) 

(1-31) 

■ (2.34) 

(3.91) 

(4.17) 

Ripe Oats 

2.67 

3.56 

3.51 

4.90 

6.26 

7.01 

7,54 

7.23 

14.86 

21.66 

15.35 

(.486) 

(.456) 

(.371) 

(.538) 

(.823) 

(.805) 

(.577) 

(.528) 

(3.03) 

(3.07) 

(.924) 

Soybeans 

5.08 

5.46 

4.94 

6.73 

9.12 

8.94 

7.49 

6.03 

3.99 

48.16 

29.94 

(.953) 

(1.25) 

(1.16) 

(1.32) 

(1,24) 

(1.63) 

(.198) 

(2.13) 

(5.52) 

(5,06) 

(3.19) 

Dense Green 

2.87 

3.07 

2.78 

4.45 

6.73 

5.92 

4.03 

2.70 

41.36 

48.08 

20.56 


(.340) 

(.2961 

(.298) 

(.516) 

(.700) 

(.801) 

(.654) 

(.502) 

(3.03) 

(4.06) 

(2.62) 

Pasture 

3.68 

4.16 

3.84 

5.57 

7.51 

7.60 

6.69 

5.54 

23.06 

34.41 

20.16 


(.136) 

(.111) 

(.203) 

(.240) 

(.510) 

(.333) 

(.549) 

(.858) 

(7.11) 

(5.79) 

(2.19) 
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The reflectances of materials shown in Table 1 are in qualitative 
agreement with reflectance spectra in the Earth Pesources Spectral 
Information System (ERSIS) and with plant canopy reflectance calculations 
made by Suits [A], The calculations of reflectance are thus felt to 
be accurate. 

2.3 TURNER MODEL CALCULATIONS 

To determine the effects of changing atmosphere optical thickness 
and of varying base altitude on ground irradiance, atmospheric trans- 
missions and path radiance, calculations were made using R. Turner's 
radiative transfer model. The model calculates these radiometric 
quantities and others given inputs of solar elevation and azimuth, 
view angles, atmospheric state, and background albedo. Atmospheric 
state is specif iced as an optical thickness at various wavelengths 
for which the calculations are to be made. 

For the model calculations, the view angle considered was nadir, 
corresponding to the satellite case. The wavelengths used were the 
center wavelengths of the ERIM M-7 scanner channels, as shown in 
Table 1. The solar elevation angle was that for the Michigan data 
(5 August 1973 at 1130 EDT) . A green vegetation albedo was assumed for 
the background. We assumed various atmospheric optical depths at 
constant base altitude, then varied the base altitude at two of the 
atmospheric optical depths. Table 2 presents the base altitude cases 
considered for the atmospheric states presented in the table. The 
intent of these cases was to cover reasonable situations. Thus the 
3-160km sea level case for atmospheric visibility spans almost the 
entire range of reasonable remote sensing situations. The altitude 
variations are for an east coast US area (Case 1) and a Rocky Mountains 
area (Case 2) . 

The atmospheric state input to the Turner model for the cases 
shown in Table 2 is straightforward ~ one needs only to specify the 
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TABLE 2 


ATMOSPHERIC STATES CONSIDERED 
FOR TURNER MODEL CALCULATIONS 
(EXPRESSED AS HORIZONTAL VISIBILITY) 



Base elevation - sea level 
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horizontal visual range and the model will make the calculation of 
atmospheric optical depth at any specifier’ wavelength. For cases 
shown in Table 3, some modified form of input was required. We simulated 
the effects of varying base elevation on atmospheric parameters, assuming 
a constant atmospheric state. Because less air is contained in paths 
to base elevations above sea level than in paths to sea level, the 
atmospheric optical depth will be less, even though the horizontal 
visibility at sea level is unchanged. Thus optical depths for base 
elevations greater than sea level need to be modified over the sea 
level case. 

For the purpose of these computations, we calculated the Rayleigh 
optical depth as a direct function of the absolute atmospheric pressure 
at the altitude in question. Thus, to obtain the Rayleigh optical 
depth for a base elevation other than sea level, the sea level optical 
depth is modified by: 

"^h “ ^0 1013 

where 

T. = atmospheric Rayleigh optical depth at base elevation h 
h 

== atmospheric Rayleigh optical depth at sea level 
- absolute pressure at elevation h in millibars 

The aerosol term was calculated for varying altitudes by reference 
to the Elterraan Standard atmosphere [2] . This standard atmospheric 
profile is the one used in the Turner model when atmospheric visibilities 
are used to characterize the atmospheric state. Then the total optical 
depth was calculated as the sum of the aerosol term, the Rayleigh term, 
and a term representing ozone absorption. 
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TABLE 3 

BASE ALTITUDE CONDITIONS FOR TURNER 
MODEL CALCULATIONS 


Case 1 — 10km 'orizontal visibllit 


Sub Case 
A 
B 
C 
D 
E 


Base Elevation (ft) 
0 * 

2000 

4000 

6000 

8000 


Case 2 — 40km horizontal visibilit 


Sub Case 
A 
B 
C 
D 
E 


Base Elevation (ft) 
0 * 

6000 

8000 

10000 

12000 


’^'represents cases shown in Table 2 
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2.3.1 DISCUSSION OF RESULTS 

After the optical depths had been calculated for each case, the 
Turner model was exercised on the IBM 370 computer. Results of the 
calculations are shown in Figures 4 and 5. Figure 4 shows the reflectance 
signature multiplicative term, and the additive term as a function 

of visibility from 3 to 160km visibility. Curves for 0.52 and 0.66 um 
are shown to illustrate the different behavior of these terms for 
different wavelengths. As can be seen from Figure 4, the multiplicative 
term Increases as visibility increases. This occurs because the path 
transmission from ground to satellite increases as the visibility 
increases. The total irradiance remains nearly constant. In Figure 4, 
the path radiance decreases as the atmospheric visibility increases. 

This occurs because there is less scattering at high atmospheric 
visibility than at low visibilities. 

Figure 5 shows the variations of signature multiplicative and 
additive terms with base altitude at an atmospheric visibility of 10km. 
Again, the curves for 0.52 and 0.66 pm are shown. Referring to 
Figure 5, as base altitude increases, the horizontal visibility 
increases. The horizontal visibility increases because there is less 
scattering at elevations above ground level. Thus the multiplicative 
term increases with base altitude both because irradiance increases and 
because path transmission increases. Referring to Figure 5, the path 
radiance term decreases as base elevation increases because of reduced 
scattering . 

Because the multiplicative term increases with increasing visibility 
or base altitude and the additive term decreases, the signature means 
for various classes may either increase, decrease, or remain nearly 
constant, depending on the reflectance in a particular band. Low 
reflectance class means will decrease with increasing visibility or 
base altitude because of the dominance of the additive term. The 
converse is true of high reflectance classes because of the dominance 
of the multiplicative term. 
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2.3.2 SIGNATURE TRANSFORMATION 

Reflectance signatures calculated from aircraft data were trans- 
formed, using the Turner model calculations previously discussed, to 
create radiance signatures under various atmospheric visibility and 
base altitude conditions. Radiance signatures are extracted from sensor 
data . 

Three types of signature transformation were employed. First, 
signature mean values were converted from reflectance to radiance by 
the formula: 


L 


Eip 

TT 


L 

P 


C3) 


where 

L = signature radiance mean [mw/cm^-ster-ym] 

E = total downward irradiance from Turner model [mw/cm^-ster-um] 
T == path transmission from ground to sensor 
p = signature reflectance mean value 
L path radiance calculated by Turner model [mw/cm^-ster-ym] 


All variables in equation 3 were functions of wavelength, and the 
calculation was carried out for each of the scanner bands, using 
Turner model values calculated at the center wavelength of the bands. 

The covariance terms of the reflectance signatures were trans- 
formed by the formula; 


V 

mn 


E E T r 
m n m n 

P — — 

mn ir^ 


(A) 


where 

V = the covariance between the radiance in spectral bands ra & n 
mn 

p = the covariance between the reflectance in spectral bands m & n 
ran 


E = total irradiance in band ra 
m 

= path transmission in band m 
m 
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noise of the sensor was introducpd by the formula: 

= the transformed variance of channel K 

= the radiance variance of channel K, calculated from 
equation 4 

= the mean square sensor noise as deduced from performance 
reports 

The simulation of scanner noise assumes that the scanner noise is 
uncorrected with variations in signature radiance. 

2.4 SIGNATURE MODIFICATION MODEL DEVELOPMENT AND EXERCISE 

2.4.1 MODEL DEVELOPMENT 

As stated in the introduction to Section 2, the approach taken to 
study of atmospheric effects on pattern recognition devices was 
modelling. Computer software for the 7094 computer was developed to 
read a signature deck and generate a sample of specified size from 
that distribution. Then the samples are classified, using the ERIM 
linear classifier [5] trained by signatures entered to the model. The 
model generates a confusion matrix, a display showing how points from 
the sample distributions were classified by the trained classifier, 

2.4.2 MODEL EXERCISE 

After the model had been developed and debugged, it was exercised 
using reflectance signature data from aircraft data analysis trans- 
formed by multiplicative and additive terms as previously discussed. 
Three broad cases of atmospheric effects were considered — ERTS-1 
effects (4 channels), S-192 effects (best 7 channels), and S-192 
(best 4 channels). For each case the effects of change of atmospheric 
visibility from 3 to 160km (training on data with 10km visibility) were 
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assessed. Two cases of variation in performance with base elevation 
changes were assessed — a variation from 2,000 to 8,000 ft (training on 

2.000 ft base elevation signatures) and a variation from 6,000 to 

14.000 ft (training on 8,000 ft data). The 2,000 to 8,000 ft test was 
conducted at 10km visibility (simulating east coast cases) and the 

6.000 to 14,000 ft test was conducted at 40km visibility (simulating 
west coast cases). These cases are summarized in Table 4. 

Finally, for each set of wavelengths and visibility or base 
elevation conditions two cases were computed — one with sensor noise 
(ERTS or S-192 as appropriate) and one without. Results will be 
discussed in Section 2.5. 

2.5 ANALYSIS OF SIMULATION MODEL RESULTS 

As a result of the simulation model calculations, results were 
generated for four optimum bands, the four ERTS bands, and seven 
optimum bands. Cases with and v/ithout sensor noise were generated to 
compare the intrinsic separability of the classes v;ith the actual 
separability under ERTS and S-192 conditions. 

2.5.1 FOUR OPTIMUM BAND RESULTS 

The four optimum channels, selected on the basis of the separation 
of the reflectance signatures, were 0.48 - 0.52, 0.55 - 0.60, 

0.62 - 0.70, and 1.5 - 1.8 ym. 

Figures 6 and 7 show results of the simulation of classifier 
accuracy versus visibility for noise free and noisy cases respectively. 
Referring to Figure 6, the probability of correct classification is 
shown as the solid line. The distance between the solid and dashed 
line shows the probability of misclassif ication . The difference 
between the dashed line and the horizontal line at 1.0 represents the 
amount not classified. For these plots, a five class recognition 
problem was assumed. The classes were corn, woods, soil, soybeans, 
and other; these classes represent those which one would typically 

classify in satellite data collected over Michigan in August. 
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TABLE 4 

CASES CONSIDERED FOR 
CLASSIFICATION MODEL EXERCISE 


Case 1 — Effects of Atmospheric Optical Thickness 

a. Select 4 bands closest to ERTS 

b. Train on 10km data 

c. Classify 3, 6, 10, 20, 40, 160km data 

Case 2 — Effects of Base Altitude-1 

a. Select 4 bands closest to ERTS response 

b. Train on 2,000 ft data 

c. Classify 2000, 4000, and 6000 cases 

Case 3 Effects of Base Altitude 

a. Select 4 bands closest to ERTS response 

b. Train on 8,000 ft data 

c. Classify 6000, 8000, lOOOO, and 12000 ft data 
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Referring to Figure 6, the average probability of correct classi- 
fication at the training condition at 10km data is 0.942 for the five 
classes. The probability of correct classification decreases as 
conditions vary from the training condition. For small changes from the 
training condition, the major effect is to assign samples to the 

"not classified" category, rather than to classify. 

Comparing Figure & to Figure 7, we see that when sensor noise 
effects are also simulated, the performance degrades from 0.942 to 
0.719 probability of correct classification at the training condition. 

But for the "noisy" case, the reduction in performance as conditions 
different from the training case are simulated is less severe than for 
the "noise-free" case because the decision regions for each class are 
larger. This occurs because each signature has larger variance in each 
channel as a result of the added sensor noise. The range of visibilities 
over which the performance is degraded less than 5% from that at the 
training condition is from 8.97km to 13.9km. Later, these numbers will 
be compared with similar numbers for the 7 optimum channel case. 

Figures 8 and 9 show variations in performance as base elevation 
Is varied from 2,000 to 6,000 ft. Training was done on the 2,000 ft 
signatures. Qualitatively, the same comments made for variations of 
visibility apply to the variation of altitude. At 10km visibility, a 
variation of base elevation from 2,000 to 2,390 ft causes a degradation 
of performance of 5% over the training condition. 

Figures 10 and 11 show similar curves for variations of altitude 
from 6,000 to 14,000 ft at 40km atmospheric visibility. In this case, 
performance is nearly constant over the range of altitudes. Slightly 
better performance is obtained away from the training condition 
because of improvement in the recognition of the "corn" and "other" 
classes. Improved recognition results because these classes were 
composites of two and four separate training sets respectively. As 
altitude varied, the recognition accuracy of each of the subclasses 
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and the misrecognitions of one subclass as another varied in such a 
way as to make the overall accuracy inprove slightly for conditions 
different from the training condition. This would not have occurred 
if these had been single training sets for corn and other. 

2.5.2 FOUR CMNNEL ERTS RESULTS 

Figures 12 and 13 show the noise-free and the noisy simulations 
of recognition accuracy for four bands simulating the ERTS-MSS 
response. These bands were 0.52 - 0.57, 0.62 - 0.70, 0.67 - 0.94, 
and 1.0-1. 4 pm. The wide bandwidth ERTS-MSS channels were only 
partially simulated by the narrower M-7 scanner spectral bands. An 
attempt was made to select M-7 bands close to the ERTS-MSS bands. 

Because the wide spectral bandwidth was not perfectly simulated, the 
ERTS results to be discussed in this section should be viewed with 
caution. Figures 12 and 13 show the same qualitative behavior as 
Figures 6 and 7. As we move av'uy from the training conditions, the 
average probability of correct classification monotonically decreases 
and the average probability of misclassif ication increases. The 
performance at the training condition for ERTS and four optimum bands 
was nearly the same, under noise-free conditions, 0.938 and 0.942 
probability of correct classification, respectively. But when sensor 
noise was simulated the performance of the ERTS was 0.817 while the 
performance of the four optimum S-192 bands was only 0.719. This 
is a consequence of the relatively lower noise levels in ERTS as 
opposed to S-192 data. 

The behavior of performance or visibility is varied away from the 
training condition is similar for the ERTS and four optimum channel 
cases. Visibility variations from 8.97km to 13.9km produced 5% 
degradations in the performance of the classifier for four optimum 
channels. The corresponding numbers for the ERTS case are 9.44 - 12.7km. 
The slightly larger range of acceptable performance of the four optimum 
channels is bought at the expense of lower performance at the training 
condition. 
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Figures 14 and 15 show the variation in performance with four 
simulated ERTS bands as the altitude is varied from 2 >000 to 6,000 ft 
at a visibility of 10km. These curves are comparable to Figures 
8 and 9 for the four optimum channels. Again, the qualitative behavior 
of average probability of correct classification as conditions are 
varied away from the training condition is similar for ERTS and four 
optimum channel cases. For the four optimum band case, an altitude 
variation from 2,000 to 2,390 ft caused a 5% decrease in performance. 

An altitude change from 2,000 to 2,760 ft would cause a 5% decrease 
in the performance using the four ERTS bands. One reason why a greater 
altitude variations are possible with ERTS than with the four optimum 
bands might be that the ERTS bands are confined to the green, red, 
and near-infrared portion of the spectrum, while the four optimum bands 
include one band in the blue-green where path radiance is a large 
fraction of the observed signal and where variations with altitude are 
large. Further, most vegetation has low reflectance in the blue-green 
as a consequence of chlorophyll absorption. Hence path radiance 
changes will lead to large changes in signature mean values for 
vegetation classes. 

Figures 16 and 17 show the variation in performance as altitude is 
varied from 6,000 to 14,000 ft at a visibility of 40km. Compare these 
figures with Figures 10 and 11 for the four optimum bands. As with the 
four optimum band case less than 5% performance degradation is observed 
over most of the range. Again the performance increases, then decreases 
as we increase altitude from 8,000 to 14,000 ft. The reason for this 
behavior \‘ia& discussed in Section 2.5,1. 

For the noise-free case for four optimum channels the performance 
decreased smoothly as elevation was increased from 8,000 to 14,000 ft. 
This behavior is noticed to a much smaller degree in the ERTS data. 

Again the difference is probably caused by the blue-green band being 
one of the four optimum bands, but not one of the four bands used to 
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simulate ERTS. For this band changes in atmospheric effects (primarily 
path radiance for dark vegetation targets) will be relatively larger 
for a given terrain elevation or visibility change than for the green 
or red bands of ERTS. 

Comparing the simulated performance of the four simulated ERTS 
bands with that of the four optimum bands, the ERTS bands show better 
performance at the training condition than the four optimum bands 
primarily because of lower noise levels simulated for the ERTS sensor. 
Performance degradations with visibility changes are similar for the 
two four band sets. Changes with elevation differences are slightly 
smaller for ERTS bands than for the four optimum bands, the latter set 
with a blue-green band. 

2.5.3 SEVEN OPTIMUM BAND RESULTS 

Figures 18 and 19 show the classifier performance in noise-free 
and noisy cases for seven optimum channels. In this case, the seven 
channels were 0.48 - 0.52, 0.50 - 0.54, 0.55 - 0.60, 0.&2 - 0.70, 

0.67 - 0.94, 1.0 - 1.4, and 1.5 - 1.8 pm, and the noise simulated was 
for the S-192 sensor. Comparing the performance of the seven channels 
with that of the four optimum or four ERTS simulated channels, the 
performance at the training condition is slightly better (0.881 average 
probability of correct classification versus 0.719 and 0.817 respectively 
for the noisy cases and 0.988 average probability of correct classifi- 
cation versus 0.942 and 0.938 respectively for the noise-free cases). 

For the noise-free case, the degradation in performance as conditions 
are varied from the training condition is more drastic for the seven 
channel case than for either of the four channel cases. This makes 
sense intuitively because the decision regions associated with each 
signature are proportionally larger for seven channels than for four 
channels, but the additional three channels (.50 - .54, .67 - .94, and 
1.0 - 1.4 pm) have larger variations in atmospheric effects with 
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changing visibility than the optimum four. Hence a given visibility 
change will induce a greater change in performance for seven channels 
than for four. 

For the noisy case, the degradation in performance as visibility 
decreases is also more severe for the seven channel case than for 
either of the four channel cases. But as visibility improves from 
the training condition of 10km, the seven channel results are less 
drastically affected than the four optimum channel results, until we 
reach the limiting case of 160km visibility. 

As we saw with the four channel results, the major effect in the 
noise-free case is for correct classification probability to decrease 
and the not classified category to increase. For the noisy case, the 
probability of correct classification decreases and the misclassifi- 
cation increases. 

If we accept a 5% decrease in classifier performance, then the 
range of visibilities over which we can operate with the seven 
optimum channels is 9.8km to 10* 58km with the noise-free data and 
9.74km to 13.3km for the case with sensor noise included. These ranges 
are narrower than for the 4 optimum or the 4 ERTS channels. This 
implies that the performdnce is more sensitive to atmospheric 
visibility variations when seven channels are used than when four 
channels are used. 

A more realistic comparison of performance between the three 
cases is to specify a certain level of performance of all classifiers 
and then calculate the range of conditions over which the classifiers 
can deliver that performance. If we somewhat arbitrarily select 65^ 
accuracy on the training sets, then the range of visibilities over 
which we can obtain this performance is as shown in Table 5. 

Table 5 contains much interesting information about the variation 
in classifier performance with visibility. First, for the noise-free 
case, note that the four ERTS bands have the widest range of applicability. 
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TABLE 5 


RANGE OF VISIBILITIES OVER IfflICH 
65% TRAINING SET ACCURACY CAN BE ACHIEVED 
(Training at 10km Visibility) 

(2000 ft base elevation) 

Upper Visibility Lower Visibility 


Case 

Limit (km) 

Limit (km) 

4 OPT Vnoise free) 

16.9 

8.74 

4 ERTS (noise free) 

19.8 

8.44 

7 OPT (noise free) 

13.9 

8.63 

4 OPT (sensor noise) 

14.6 

8.79 

4 ERTS (sensor noise) 

19.02 

8.13 

7 OPT (sensor noise) 

36.4 

8.76 
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This occurs because the perfortnance at the training condition is good 
and the performance does not degrade too rapidly with changes in 
visibility because the four bands used to simulate ERTS are far enough 
into the red to avoid most of the major changes in atmospheric effects 
with visibility, which occur in the blue. By comparison, the range of 
the four optimum bands is smaller than the four ERTS bands for the 
noise-free case. Better performance at the training condition is 
offset by greater variation in performance away from the training 
condition because the four optimum bands contain a blue-green band 
more sensitive to atmospheric effects than the ERTS bands. Thus 
while the four bands are optimum for separating the classes at the 
training condition, they may not necessarily be optimum for signature 
extension over visibility changes. The seven optimum channel noise- 
free results show greater sensitivity to atmospheric visibility 
changes than the four optimum channel results. Here too, the increased 
performance at the training condition is offset by greater sensitivity 
to atmospheric effects which change with visibility. Thus the number 
of channels used for classification under conditions where signatures 
must be extended over visibility variations must be selected with 
signature extendability , as well as classifier performance on the 
training sets, in mind. 

Comparing the cases where sensor noise was also simulated, the 
four ERTS bands still have a larger range of permitted visibility 
variations for a performance of 65% than the four optimum bands. The 
lower visibility limit is slightly lower for the noisy ERTS case than 
for the noise-free case because the performance under noisy conditions 
falls off more slowly with decreasing visibility than under noise-free 
conditions. The four optimum band case has the most restricted operating 
range under conditions where S-192 sensor noise was simulated. The 
seven optimum band case has the widest range of operation under these 
noisy conditions, mainly because of the slow degradation of performance 
with increasing visibility. 
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Figures 20 and 21 show the variation of classifier performance 
for seven optimum bands, with and without sensor noise respectively, 
as base elevation is varied from 2,000 - 6,000 ft for a visibility 
of 10km, For both cases, the performance decreases as we move away 
from the 2,000 tL training altitude. The effect is more drastic for 
the noise-free case than for the noisy case. As we move away from the 
training altitude, the major effect in the noise-free case is a 
reduction of correct classification and an increase in not classified. 
Under S-192 noise conditions a more gradual decrease in correct 
classification occurs and an increase in misclassified. Similar effects 
were noted when visibility was varied away from the training conditions. 

Table 6 summarizes the range of altitudes over which or better 
average correct classification can be achieved for the various cases. 

The same qualitative comments apply to this table as to the previous 
table where visibility ranges t^ere tabulated. For the noise-free case, 
the four ERTS bands show the largest altitude range for "acceptable" 
performance. The four optimum bands show a reduced altitude range 
because of the blue-green band being one of the four optimum. The 
seven band case shows the smallest altitude range. For the noisy case, 
the seven band case shows the largest altitude range and the four 
optimum band case the least altitude range. The four ERTS band case 
is intermediate. 

Figures 22 and 23 show the variation in classifier performance 
with base altitude changes at 40km visibility for seven optimum bands 
for noise free and noisy cases respectively. As with the two 
four channel cases there is a decrease in correct classification and an 
increase in misclassif ication as we move away from the 8,000 ft training 
altitude. But for the seven channel case the degradation in performance 
is more drastic for the noise-free case than for the noisy case, in 
contrast to the four channel results. 
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TABLE 6 


RANGE OF BASE ALTITUDES OVER WHICH 
65% TRAINING SET ACCURACY CAN BE ACHIEVED 
(10km Visibility, Training at 2,000 ft) 


Case 

4 OPT (noise free) 

4 ERTS (noise free) 

7 OPT (noise free) 

4 OPT (sensor noise) 

4 ERTS (sensor noise) 
7 OPT (sensor noise) 


Upper Altitude 
Limit (kft) 

3.08 

4.52 

2.72 

2.67 

5.21 

6 . 0 * 


Lower Altitude 
Limit (kft) 

2.0 

2.0 

2.0 

2.0 

2.0 

2.0 


*performance not degraded to 65% at 6.0kft 
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Using the 65% or greater correct classification criterion, 
acceptable performance is obtained over the entire range of altitudes 
for all cases except the seven channel noisy case, where the specified 
performance can be obtained only from 6,000 to 12,000 ft. 

2.5.4 MODEL VERIFICATION WITH ACTUAL SKYLAB PROCESSED RESULTS 

In an effort to verify that the model results (particularly those 
results where sensor noise was simulated) were accurate, we processed 
actual S-192 data collected over the Michigan Test Site on 5 August 1973. 

The details of processing are discussed in Sections 2.3 and 2.6. The training 
sets selected from the S-192 data were the same as those selected from 
the aircraft data from which the reflectance signatures were obtained. 

T’^aining and test set performance were evaluated. These results are more 
fully discussed in Section 2.6. However, Table 7 presents a comparison 
of training set accuracies from the simulated S-192 data and the actual 
S-192 data. The 2,000 ft base elevation and 10km visibility conditions 
are reasonably close approximations to the actual conditions at the 
Michigan Test Site during the Skylab overpass [3]. As can be seen from 
Table 7, the agreement between the two results is good, particularly 
when the 90% confidence intervals of the estimate from the actual data 
are considered. These confidence intervals are important because of the 
relatively small number of points (435) used for the evaluation of actual 
S-192 data. All simulated results fall within the 90% confidence intervals 
of the actual results, except corn, soybeans, and the average. Differences 
may be attributed to the slight difference in the actual and simulated 
spectral bandwidths, the slight difference between the simulated 
conditions and the actual conditions, and to minor variations in actual 
S-192 sensor noise from that tabulated in reference 4. The fact that 
signatures are not really gaussian, but were assumed so for the 
simulation is also a factor which must be considered. But on the 
whole, the agreement between simulated and actual seven optimum band 
S-192 results is good, lending credibility to the conclusions to be 
drawn from the simulation study. 
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TABLE 7 


CO^^PARISON OF SIMULATED S-192 AND 
ACTUAL S-192 TRAINING SET CLASSIFICATION 
ACCURACY RESULTS FOR 2,000 ft BASE 
ALTITUDE AND 10km VISIBILITY 


Class 

Simulated Accuracy 

S-192 Accuracy' 

Corn 

83.4 

73.8 

+ 

8.1 

Woods 

91.1 

78.6 


3,7 

Soil 

90.7 

90.5 

± 

11.3 

Soybeans 

86.0 

68.1 

+ 

17.0 

Other 

89.6 

90.0 

± 

7.2 

Average 

88.2 

80.2 

+ 

4.8 


*inean value + 90% confidence interval 
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2,5.5 CONCLUSIONS FROM THE SIMULATION STUDY 

Conclusions reached from the simulation study are reported in 
this section. When reading these conclusions, the reader should be 
aware that this is but one study, using one set of data of one 
agricultural scene. The conclusions drawn are supported fay the 
results of the experiment. Generalization to a broader agricultural 
mapping problem should bf. ^one with care, and with due regard for other 
studies relating to this general problem (e.g., reference 6). 

Under training conditions, and noise-free conditions, the seven 
optimum channels perform better than the four optimum and four simulated 
ERTS bands (0.988 versus 0.942 and 0.938 average probability of correct 
classification). But the seven optimum channel results are more 
sensitive to changes in atmospheric visibility (Table 5) or base 
elevation (Table 6). When S-192 or ERTS sensor noise is simulated 
this behavior is not as dramatic. 

Classification using the four narrow aircraft data bands to 
simulate ERTS bands is less affected, in either noisy or noise-free 
state, by changes in atmospheric or base elevation conditions than 
classification using the four optimum bands. This is probably because 
a blue-green band is one of the four optimum bands, and variations in 
atmospheric effects in this band are larger than for the simulated 
ERTS bands . 

The performance using narrow wavelength bands to simulate ERTS, 
and four optimum channels, are nearly the same under noise-free 
conditions (0.942 versus 0.938 average probability of correct 
classification). Under conditions of sensor noise, the four simulated 
ERTS bands show better performance than the four optimum bands, probably 
because the noise levels from the ERTS sensor are generally lower than 
from the S-192 sensor. 

The seven channel results with sensor noise agree well with actual 
performance on the same fields from the S-192 processing. Small 
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differences in performance may be attributed to slight mismatch of 
simulation study bands and actual S-192 bands » to slight differences 
between simulated and actual atmospheric conditions, and to lack of 
strictly gaussian training set statistics. The good agreement between 
simulated and actual results verifies that the noise conditions used 
for S-192 were representative of the noise in the data. 

The implications for use of ERTS and S-192 like sensors for 
agricultural surveys relate to the range of atmospheric variability 
over which acceptable performance can be achieved. At atmospheric 
conditions equivalent to horizontal visibilities of about 10km, 
adequate performance (65% correct classification or better) can be 
obtained over atmospheric state variations corresponding to visibility 
variations of + 3 “ 4km. If greater variations are found, data 
preprocessing corrections will have to be made to assume adequate 
performance. Although training at other than 10km visibility 
conditions was not performed, it is expected that as the visibility 
improves, the range of visibilities over which adequate performance 
can be obtained will increase. 

The implication for the use of future spacecraft sensors, with 
improved slgnal-to-noise ratio is contained in a comparison of noisy 
versus noise-free results. The noise-free results (as would be 
generated with a sensor with no noise) show better performance at the 
training condition but greater sensitivity to changes in atmospheric 
condition or base elevation than the noisy results. As better sensors 
are constructed, there should be more concern about selecting a set of 
bands and a number of bands to use for classification. The bands 
selected and the number of bands can only be partially specified by 
results of optimum channel studies and training set results generated 
under a limited set of atmospheric conditions. Due consideration must 
also be given to the range of atmospheric conditions over which surveys 
will operate. For example, in the present study, seven optimum channels 
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(noise-free) gave better performance at the training condition than 
four optimum channels, but the seven channel results were more sensitive 
to changes in atmospheric visibility than the four channel results. 

Which set would perform better in practice would depend on the user's 
definition of acceptable performance, but for 65% accuracy or better, 
the four channel results showed a wider range of visibilities than the 
seven channel results. 

2.6 S-192 DATA HANDLING 

The purpose of processing the S-192 data was to generate a 
recognition map comparable to the aircraft data recognition map and to 
evaluate its accuracy for comparison with the aircraft data and with 
the model calculations. The data used were SL-3 S-192 data collected 
over the Williamston, Michigan Test Site on 5 August 1973 at about 
1100 EDT. 

Referring to Figure 24, the first step in processing was to 
convert the S-192 data into 7094 computer format through the use of an 
IBM 360 conversion program. Then a reconnaissance grayraap of the 
1.5 - 1.8 pm band (every 10th line and point) was made to edit the area 
to that of the test site. 

Next a 1 X 1 gray scale optimized map of the 1.55 - 1.75 pm band 
was prepared to select training sets, and the same training sets selected 
for aircraft data analysis were located. Then signatures were extracted 
for these fields from the S-192 data. As before, at least two samples 
of each of seven agricultural classes were selected, and the samples 
of each class combined to form the signature for that class. 

A recognition map was prepared, using the optimum seven bands 
as selected from the analysis of aircraft data [6], The bands used 
are shown in Table 8. When the recognition map had been prepared, its 
accuracy was evaluated on both the training set and the test set. The 
test set selected was different from the one used for the aircraft data 
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TABLE 8 


SEVEN OPTI^m BANDS USED 
FOR S-192 COMPUTER MAPPING OF 
MICHIGAN AGRICULTURE 


0,46 

- 0.51 

Ijm 

0.52 

- 0.56 

yra 

0.56 

- 0.61 

ym 

0,62 

- 0.67 

um 

0.78 

- 0.88 

um 

0.98 

- 1.08 

Vim 

1 , 55 

- 1.75 

Vim 
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evaluation [6] because it was impossible to find some of the small 
fields of the aircraft test set on the S-192 data. 

2.6.1 TRAINING AND TEST SET LOCATION 

Both training and test sets were difficult to find on the S-192 
data graymap. This was caused by two factors — the resolution of the 
data and its radiometric quality, and the fact that conical scrn data 
were used. (Scan line straightened data was originally requested, but 
was not used because other investigators had found mis-registration 
between the various channels of data in the scan-line straightened 
data [7 ] . ) 

To accurately locate both training and test sets, we first prepared 
a grid overlay to the graymap showing all section lines. This grid 
was prepared by a two-step procedure. First, control points were 
located in both S-192 and aerial photographic data. Then section 
corners were measured from the aerial photography, using the same 
coordinate system as for the control points. A regression equation 
related the coordinates of the section corners in the photography to 
the line and point numbers on the S-192 data. The regression equation 
was developed from the control points. 

The resultant section grid was only approximate, because of 
residual errors in the regression equations relating aerial photograph 
coordinates to S-192 data coordinates. Nevertheless, the grid was 
adequate for location of the training sets. A detailed comparison of 
the field pattern within a particular section grid and the pattern on 
the aerial photography allowed us to locate specific training and test 
fields which had been annotated on the photography. 

2.6.2 EVALUATION OF MAP ACCURACY - TRAINING SETS 

The evaluation of map accuracy was done on both training and test 
sets. For training set accuracy assessment, the actual classification 
within the training areas was counted. Initially, recognition as nine 
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separate categories were assessed. The recognition results were later 
aggregated to the five classes at interest in crop mapping surveys: 
corn, woods, soil, soybeans, and other. Some of the results of the 
analysis have already been presented in Table 7. Table 9 presents a 
confusion matrix of training set results showing not only percentage 
correct classification but how the various misclassif ications are 
apportioned. In Table 9, the ground classes are shown across the top 
of the table, while the classifier classes are shown down the left hand 
side. For example, 73.8% of corn points were called corn by the classifier. 

The classifier called 10,6% of the corn woods, 10.0% soybeans, and 5.6% 
other. The numbers in parentheses show the standard deviation of the 
estimate of classification accuracy, computed assuming a binomial distribution. 

Referring to Table 9, we see that corn is misclassif led as woods, 
soybeans and other. The corn-woods misclassif ication, and corn-other 
misclassif ications have been seen by other investigators using satellite 
and aircraft data [8,9]. The misclassif ication of corn as soybeans 
apparently occurs in sparsely vegetated areas of corn (drowned spots or 
field edges) more representative of soybeans than corn. 

There is some confusion of bare soil with other, primarily stubble 
and pasture. There may be some vegetation growing in the bare soil 
fields used for training, and this would account for these misclassi- 
f ications . 

There Is misclassif icat ion of woods as corn and soybeans. The 
woods corn misclassif ication has been mentioned before. Some sparse 
woods may be classified as soybeans. 

The soybeans are confused x^ith every other category except woods. 

This probably occurs because the soybeans had variable percentage cover 
of the ground at the time of overflight. Ordinarily this would not 
have occurred at this point in the growing season but 1973 was a very 
late year for crops as heavy spring rains delayed planting. Thus the 
soybeans were at a stage of development more typical of mid- July than 
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TABLE 9 


COKr'USION MATRIX FOR AGRICULTURAL CLASSIFICATION 
WITH SEVEN S-192 CHANNELS - MICHIGAN TEST SITE 8/5/73 

Training Set Results 



Corn 

Soil 

Woods 

Soybeans 

Other 

Corn 

73.8 

(3.27) 

0 

14.3 

6.4 

3.6 

Soil 

0 

90.5 

(4.52) 

0 

2.1 

1.8 

Woods 

10.6 

0 

78.6 

(5.48) 

0 

0 

Soybeans 

10.0 

0 

7.1 

68.1 

(6.8) 

4.6 

Other 

5.6 

9.5 

0 

21.3 

90.0 

(2.88) 

Not 

Classified 

0 

0 

0 

2.1 

0 


Percentages of correct and mis-classlf ication are shown. Numbers in parentheses 
are the standard deviations of the estimates. 
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early August, Very dense cover soybeans were misclassified as corn, 
''normal” cover areas were called soybeans, vi?hile sparse cover areas 
were called soil, other, and not classified. This kind of result was 
also seen in the test results. It is attributed to the delayed 
development of the soybean crop at the time of overflight in the summer 
of 1973. 

The other class consisted of pastures, alfalfa and hay fields, 
and idle fields. While there were many misclasslf ications between the 
four signatures used to recognize this class, the recognition of the 
class as a whole was about 90%, The largest misclassif ication was as 
soybeans. The reason for this has been discussed above. 

2.6.3 EVALUATION OF MAP ACCURACY - TEST SETS 

Evaluation of accuracy on training sets frequently gives an 
artificially good estimate of the actual classifier performance. For 
this reason, test set evaluation of classifier performance was dona. 

Table 10, shows the results of this evaluation for the four classes con,', 
woods, soil, and soybeans. The results are presented in the same format as 
Table 9. The results of Tables 9 and 10 are comparable — that is the 
accuracy results are within one standard deviation of each other — for 
all classes except soybeans. Comparing the soybeans results, we find 
larger misclassif ication as other and corn in the test set than in 
training set results. In view of the vari bility of the soybeans at 
this stage in the growing season, this result should be expected. 

Also, there are relatively few soybeans fields in the test area, and 
those fields are small. Thare was great difficulty in locating the 
soybeans test set fields. 

On the whole, the results are as good as those being reports by 
other investigators [8] . They are probably representative of I'jhat can 
be done with 3-192 data under these agricultural conditions. In a more 
normal year, we might expect improved performance on the soybeans 
category . 
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TABLE 10 


CONFDSIOK MATRIX FOR AGRICULTURAL CLASSIFICATION 


WITH 

SEVEN S-192 

CHANNELS - 

- MICHIGAN TEST 

SITE 8/5/73 



Test Set 

Results 



Corn 

Soil 

Woods 

Soybeans 

Corn 

71.1 

(4.97) 

0 

11.6 

14,3 

Soil 

0 

S5.7 

(5.04) 

0 

0 

Woods 

22.9 

0 

81.4 

(5.93) 

0 

Soybeans 

0 

7.0 

4.8 

50.0 

(7.71) 

Other 

6.0 

9.5 

0 

35.7 

Not 

Classified 

0 

0 

0 

0 


Percentages of correct and mis~classif ication are shown. Numbers 
in parentheses are the standard deviations of the estimates. 
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S-192 LAND RESOURCE MAPS OF CRIPPLE CREEK AREA 

As a cooperative program with Dr. Harry Smedes of the U. S. 
Geological Survey, Denver, Colorado, and Mr. Jon Ranson, a Colorado 
State University graduate student, we prepared land resource maps of 
the Cripple Creek area of Colorado. The project was similar to one 
conducted with ERTS data [10] and resulted in a seven category computer“ 
generated land resource map prepared using six optimum channels of 
S-192 data collected on 4 August 1973 at about 1719 hrs. GMT on the 
SL-3 mission. 

3.1 TEST SITE DESCRIPTION 

Figure 25 shows a portion of an S-190A red band (0.6 - 0.7 pm) 
frame with the test site outlined. Also shown is the outline of the 
ERTS project test site of [10], S-192 data were collected over this 

site on various occasions, but the data from 4 August 1973 were selected 
for study because the vegetation categories were differentiable at 
that time, there was a minimum of snow cover, and the data were collected 
reasonably close to the time of the ERTS data previously processed. 

The area of Figure 25 covers a portion of south-central Colorado 
centered on the town of Cripple Creek. The area, about 20 x 25 miles 
in extent, extends from nearly Eleven Mile Canyon Reservoir on the 
northwest to beyond Colorado Springs on the east. The western half of 
the test site is mountainous terrain, while the eastern half is Great 
Plains. Pikes Peak is in the upper middle of the test site. Elevations 
within the test site vary from 5,000 ft for the Great Plains area to 
over 14,000 ft on Pikes Peak. Cripple Creek is about 10,000 ft 
elevation. The mountainous area consists of bare rock, coniferous 
forest over a variety of rock substrates, and grassland meadows. The 
Great Plains area consists of agriculture and rangeland. The rangeland 
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has a sparse cover of grass. In between the mountainous terrain of the 
Rocky Mountains and the Great Plains, there are exposed a set of 
sedimentary rocks. These rocks were exposed when the Rocky Mountains 
were uplifted. 

3.:: APPROACH 

The approach to this land resource mapping task was to use computer- 
implemented pattern recognition processing of S-192 digital taped data. 
The flow of operations is summarized in Figure 26. We performed most 
of the computer processing at ERIM on an IBM 7094 computer. Dr. Smedes 
and Mr. Ranson assisted us by selecting the training sets and in 
providing guidance as to the final set of classes for mapping and 
display. 

We began processing with scan-line straightened computer compatible 
tapes of S-192 data. The first step was the conversion of these data 
to a format compatible with our 7094 software package. The format 
conversion was done on an IBM 370/168 computer at the University of 
Michigan. Tapes in the ERIM standard format were then histogrammed to 
provide a quantiative measure of the dynamic range of data in all 
spectral channels. As with other S-192 digital tape data sets we have 
examined, we found the dynamic range in each channel to be a factor of 
5 to 10 less than the maximum dynamic range of the tape. 

We used histograms and the imagery of the scan line straightened 
data (also provided by NASA) to perform preliminary data quality 
assessment and to select a channel for mapping and for eventual selection 
of training sets. Examination of the imagery revealed that band 1 
(0.41 - 0.48 pm) was so noisy as to be nearly useless. All other bands, 
including the thermal band (10.4 - 12.5 pm) showed adequate or good 
contrast. After careful examination of the imagery and histograms, 
band 11 (1.55 - 1.75 pm) was selected for mapping because it had the 
largest dynamic range of any of the channels, the imagery of that band 
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showed good terrain contrasts » and the relative tones of vegetation, 
soil and rock, and water were similar to those in the red spectral 
region, permitting easy comparison of red band photography and the 
1.55 “ 1.75 pm computer map. 

The 1.55 - 1.75 pm computer graymaps were sent to Dr. Smedes and 
Mr. Ranson for selection of training sets. They sometimes selected 
multiple areas to represent training sets for classes of materials 
they wished to discriminate in the data. Following the selection of 
training areas (see Table 11) , and their designation to ERIM as sets of 
line and pixel numbers in the original S-192 data, spectral signatures 
were calculated. (A spectral signature is a description of the mean 
values, variances, and covariances of the signals in each spectral band 
for a particular material.) Copies of the signature information were 
sent to Dr. Smedes and Mr. Ranson who subsequently decided to combine 
the signatures of some classes whose signatures were not sufficiently 
different to permit reliable discrimination. 

To assist them in deciding which of their original classes should 
be combined into composite classes, ERIM generated a set of pairwise 
probabilities of misclassif ication for each signature pair. The average 
pairwise probability of misclassif ication criterion was used to select 
six optimum bands for further mapping from the thirteen available S-192 
bands [11]. A 7094 computer program, STEPL, was exercised for this 
selection. Two different runs of the program were made, one with all 
22 terrain class signatures, and one with 14 signatures representing 
the bare rock and soil classes. Results of the channel selection for 
each of these cases are summarized in Table 12, with the consensus bands 
shown in Table 13. These results are further discussed in the next 
section. In selecting the bands for the consensus, where there was a 
choice, a band which appeared high in the ordering for separating rock 
types was selected, since the goal of the experiment was primarily to 
map rock types. 
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TABLE 11 

TRAINING SETS SELECTED FOR CRIPPLE CREEK MAPPING 


Designation 

Brief Description 

CCG 

Cripple Creek Granite 

DS 

Dakota Sandstone 

FOUNT 

Fountain Formation 

PHONGRAND 

Phonolite - Granodiorite 

PS 

Pierre Shale 

VOL 

Volcanic Rocks 

CCGC 

Composite of Cripple Creek Granoite and 
Coniferous Forest 

DSC 

Composite of Dakota Sandstone and Coniferous 
Forest 

GRANOC 

Composite of Granodiorite and Coniferous Forest 

PPG 

Pikes Peak Granite 

NS 

Niobrara Shale 

PPGC 

Composite of Pikes Peak Granite and Coniferous 
Forest 

VOLC 

Composite of Volcanics and Coniferous Forest 

CCGF 

Dense Coniferous Forest over Cripple Creek 
Granite 

GRANOF 

Dense Coniferous Forest over Granodiorite 

PPGF 

Dense Coniferous Forest over Pikes Peak Granite 

VOLF 

Dense Coniferous Forest over Volcanics 

CLOUD 

Clouds 

CLDS 

Cloud Shadow 

MEAD 

Meadow (grass) 

SNOW 

Snow 

WATER 

Water 
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TABLE 12 

ORDERING OF S-192 BANDS FOR TWO 
CLASSIFICATION PROBLEMS ~ CRIPPLE CREEK 

Order General Classification Rock Classification 


1 

1.55 

“ 

1.75 

0.56 

- 

0,61 

2 

0.78 

“ 

0.88 

1.55 

- 

1.75 

3 

1.09 

- 

1.19 

0.68 

- 

0.76 

4 

2.05 

- 

2.35 

1.09 

- 

J-.19 

5 

10.4 

- 

12.5 

2.05 

- 

2.35 

6 

0.62 

- 

0.67 

0.62 

- 

0.67 

7 

0.68 

- 

0.76 

10.4 

- 

12.5 

8 

0.56 

- 

0.61 

0.78 

- 

0.88 

9 

.52 

- 

.56 

0.41 

- 

0,46 

10 

1.2 

- 

1.3 

1.2 

- 

1.3 

11 

0.41 

- 

0.46 

0.52 

- 

0.56 

12 

0.98 

- 

1.08 

0.98 

- 

1.08 

13 

0.46 


0.51 

0.46 


0.51 
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TABLE 13 

CONSENSUS SIX BANDS FOR CRIPPLE CREEK 
MAPPING FROM S-192 DATA 

S-192 Channel Wavelength (ym) 


4 

0.56 

- 0.61 

5 

0.62 

- 0.67 

6 

0.68 

- 0.76 

9 

1.09 

-1.19 

11 

1.55 

- 1.75 

12 

2.05 

- 2-35 
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Following the selection of a consensus set of six bands for mapping, 
and the definition of a good set of classes for mapping (shown in Table 11), 
recognition maps of small test areas were prepared. This was done to 
permit a final assessment of the set of classes for mapping and to 
provide small test area information. After examination of the preliroi- 
narv set of recognition maps by Mr. Ranson, a final set of classes, 
shown in Table 14, was defined and a final recognition map of the whole 
area was prepared. 

Rather than display the recognition map using the conventional 
computer-printed map display, a more concise display format was sought. 

We had used the Mead Technology Labs ink jet printer to prepare concise 
display of ERTS recognition maps and considered it for the display of 
the final S-192 map. In the time between the preparation of the ERTS 
maps of reference 10 and the time we were ready to print our S-192 
maps, a less expensive ink jet printer capability was developed as part 
of ERIM's MIDAS (Multivariate Interactive Digital Analysis System) [12], 

We used MIDAS ink jet printer capability to generate the original of the 
color coded recognition map shown photographed in Figure 27. The cost 
of preparing this map was approximately $150. 

3.3 DISCUSSION 

The processing plan followed for the analysis of S-192 data was 
very similar to that for the analysis of ERTS data of the same area. 

Only the processing to identify optimum channels was added to the S-192 
data processing. In this section, the results of the optimum channel 
analysis and the nature of the final classes of the recognition map of 
Figure 27 are discussed and compared with ERTS results. 

3.3.1 OPTIMUM CHANNEL RESULTS 

Any optimum channel results generated by channel selection programs 
and using training set statistics should be viewed with some caution. 

Both sensor data quality (signal-to-noise ratio) and the nature of the 
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TABLE 14 

CLASSES OF THE FINAL CRIPPLE CREEK MAP 

Undivided Sedimentary Rocks 
Undivided Volcanic — Plutonic Rocks 
Dense Forest 
Sparse Forest 
Meadow 

Water and Cloud Shadow 
Cloud and Snow 
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FIGURE 27. COLOR 
CODED INK JET 
PRINTER DISPLAY OF 
FINAL CRIPPLE 
CREEK LANT* COVER 
MAP GENERATED 
FROM S192 DATA. 

Legend 

Green - Dense Forest 

Violet - Sparse Forest 

Yellow - Meadow 

Brown - Sedimentary 
Rocks 

Red - Volcanic Rocks 

Black - Cloud Shadow 
and Water 

White - Snow', Clouds 
and 

Not Classified 


/ 

N 
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classes being mapped control which channels are selected. Any set of 
channels used for classification should be justified not only on the 
grounds of the channel selection program results, but also considering 
data quality and the nature of the reflectance signatures of the materials 
being discriminated. Considerations of data quality are paruicularly 
important in the case of S-192 data because of the obvious variation 
in signal-to-noise ratio from channel to channel. Thus, it may very 
well be that high signal-to-noise ratio, non-optiraally placed spectral 
channels are selected before low signal-to-noise ratio optimally placed 
spectral channels because the low signal-to-noise obscures intrinsically 
larger signature separability in the optimally placed bands. This 
hypothesis must be examined before we can generalize from channel 
ordering results to say that a particular spectral band is a good one 
for a particular classification problem, 

Table 15 lists the S-192 bands in order of decreasing signal-to- 
noise ratio. The data were obtained from reference 5. In the same 
table, the ordering of these bands for general classification and rock 
type mapping are presented. The conclusion drawn from Table 15 is 
that signal-to-noise ratio does not materially influence the order 
of selection of channels. For example, although channel 2 (0.46 - 0.51 pm) 
has the best noise equivalent reflectance, it is selected last for both 
general classification and for rock type mapping. Channel 11 
(1.55 - 1.75 pm) has the ninth smallest noise equivalent reflectance 
but is selected first for general mapping and second for rock type 
mapping. Similarly channel 12 (2.05 - 2.35 pm), which has the largest 
noise equivalent reflectance of any of the reflective bands, is selected 
fourth best for general mapping and fifth best for rock type mapping. 

Of the six bands selected for the classification, all can be 
justified on physical grounds. Referring to Table 13, the six bands 
selected bear marked similarity to the four ERTS bands plus two near 
infrared bands, and to early recommendations for the EOS Thematic 
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TABLE 15 

ORDERING OF S-192 BANDS BY SIGNAL-TO-NOISE RATIO 
AND BY UTILITY IN TWO CLASSIFICATION PROBLEMS 


Order 

Siganl-to-Noise 

General 

Mapping Utility 

Rock 

1 

2 

11 

4 

2 

3 

7 

11 

3 

6 

9 

6 

4 

8 

12 

9 

5 

1 

13 

12 

6 

4 

5 

5 

7 

7 

6 

13 

8 

9 

4 

7 

9 

11 

3 

1 

10 

10 

10 

10 

11 

5 

1 

3 

12 

12 

8 

8 


13 


(13) 


2 
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Mapper [6], The bands of Table 13 are not listed in order of selection, 
but by increasing wavelength. 

The 0.56 - 0.61 um band is useful for vegetation assessment because 
it is close to the green peak (actually on the long wavelength side of 
the peak) of reflectance caused by lack of chlorophyll absorption, Thir, 
band is also useful in discriminating rock types containing ferric iron 
because of the absorption by ferric iron in the yellow-green portion of 
the spectrum. 

The 0.62 - 0.67 pm band is very useful in discriminating vegetation 
types (because it is placed near the red chlorophyll absorption dip) 
and is also useful in discriminating ferric iron containing rocks (when 
used with the 0.56 - 0.61 pm band) because of the lack of ferric iron 
absorption in the 0.62 - 0.67 pm region. 

The 0.68 - 0,76 pm band may be useful in discriminating vegetation 
types. The band is located near the "shoulder" of the vegetation 
reflectance curve, where the vegetation reflectance rises steeply. 
Changes in the position or height of the shoulder would be reflected as 
changes in the reflectance in this band. The selection of this band 
third for rock type mapping cannot be justified by any particular 
spectral features in the spectra of the rocks in the test site. 

The 1,09 - 1.19 pm band (or a band with similar response) has 
been found optimum for vegetation mapping in nearly every study 
addressing optimum bands. This band was also selected third for rock 
type discrimination, probably because the rock type training sets 
contained some vegetation as well as rock. The 1.09 - 1.19 pm band does 
not match the ferric or ferrous iron absorption features at 0.95 and 
1.0 pm respectively, and therefore cannot be justified on that basis. 

The 1.55 - 1,75 pm region has also been shown in many studies to 
be a good band for vegetation classification. In this region the 
controlling factor in vegetation reflectance is the water content of 
the foliage. This band is thus a useful one for discriminating live 
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from dead vegetation. Its selection second for rock type discrimination 
is probably a consequence of the vegetation content of most of the rock 
type training sets. 

Comments similar to that for the 1,55 - 1.75 pm band can be made 
for the 2.05 — 2.35 pm band. The conclusion of the spectral band analysis 
is that despite variations in data quality from band to band, optimum 
channels selected are well matched to major vegetation and rock type 
spectral reflectance features. Bands in the yellow, red, and near 
infrared were most useful. These bands represent narrowed, and slightly 
better placed bands than ERTS bands, plus two bands in the 1.5 - 2.5 pm 
region which ERTS did not have. The latter two bands have been 
consistently recommended as good bands in EOS Thematic Mapper studies. 

3.3.2 FINAL MAP CATEGORIES, COMPARED WITH ERTS 

After consultation between Mr. Ranson and ERIM personnel, a final 
set of classes to be displayed were selected. These classes are shoxvn 
in Table 16 along with the classes mapped from the ERTS data collected 
on 20 August 1972 and reported in reference 10. Because of a misunder- 
standing between Mr. Ranson and ERIM, the Niobrara Shale signature was 
omitted from the final recognition map shown in Figure 26. There had 
been preliminary indications that the Niobrara Shale could be reliably 
discriminated from other rock types using the S-192 data. 

Analysis of Table 16 reveals that the subjective analysis of 
recognition accuracy in test sites and the objective analysis of 
pairwise probabilities of misclassif icatlon from the channel selection 
program STEPL convinced Mr. Ranson that the sedimentary rock types, 
with the exception of Niobrara Shale, could not be accurately separated 
by the S-192 data, A quantative analysis of accuracy of recognition 
to be performed by Dr. Smedes and Hr. Ranson will reveal the accuracy 
of mapping the composite sedimentary rock class. 

Further, the Intermediate forest class, consisting of intermediate 

cover coniferous forest over various rock type substrates, was not formed 
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TABLE 16 


FINAL MAP CLASSES FOR ERTS AND S-192 
COMPUTER MAPPING OF CRIPPLE CREEK TEST SITE 


S-192 


Niobrara Shale 


(Niobrara Shale) 


Pierre Shale 


Limestone 


Undivided Sedimentary 
Rocks 


Dakota Sandstone 


Fountain Formation 


Grass 


Pikes Peak Granite 


Forest 


Forest 


Intermediate Forest 


Undivided Volcanic 
Rocks 

Meadow 


Water 


Cloud Shadow 


Cloud Shadow 


(Cloud and Snow) 


Cloud and Snow 
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for the S-192 results. Instead these categories were lumped into either 
the undivided volcanic-plutonic rocks or the Pikes Peak granite classes, 
depending on the substrate. It was judged not possible to separate the 
various "exposed" volcanic-plutonic rocks with ERTS data, and these 
classes were lumped into a composite grassland category. Pikes Peak 
granite was judged separable from the other volcanic-plutonic rocks 
(granodiorite , phonolite. Cripple Creek granite, and volcanics) in the 
S-192 data. 

The other categories were similar between the ERTS mapping and 
S-192 mapping efforts. The water category probably could have been 
separated from the cloud shadow category, but the same misunderstanding 
which caused the omission of the Niobrara Shale signature caused the 
omission of the water signature in the final classification results. 

3.3.3 QUANTITATIVE CLASSIFICATION ACCURACY OF THE 
S-192 RECOGNITION MAP 

Quantitative recognition accuracy of the map will be assessed by 
Dr. Smedes and Mr. Ranson, Because accuracy figures were not available 
at the time this report was written, we are unable to include these 
numbers. 

3.4 CONCLUSIONS 

Several conclusions were reached as part of this study of land 
cover mapping in the Colorado Mountain Area near Cripple Creek. First, 
six useful channels for discriminating vegetation and and rock classes 
v'ete selected. These bands were narrowed versions of the four ERTS 
bands plus two bands In the near infrared region from 1.55 - 2.35 pm. 

Most of the bands selected can be justified on the basis of the 
reflectance spectra of vegetation and rock types. The lack of correlation 
of channel selection ordering with noise equivalent reflectance in each 
band is indicative that reflective band data quality did not play a 
major role in the selection of channels. 


fORM£RLV WILLOW RUN LA BORATOF?]^S, tH£ UNIVERSrTV Or MICHIGAN 


There was preliminary evidence that sedimentary rock type 
discrimination could be done more precisely with ERTS data than with 
S-192 data, perhaps because of the relatively higher noise equivalent 
reflectances of the S-192 channels, A final conclusion regarding this 
hypothesis must await quantitative accuracy evaluation of the S-192 
recognition map to be performed by Dr. Smedes and Mr, Ranson, 

In spite of a time of data collection near solar noon, the thermal 
channel was only the fifth most useful band for general classification 
(all scene categories) and seventh most useful for rock type classi- 
fication. This result is at odds with studies made using aircraft data 
(e,g., reference 6) and may be explained by the relatively large 2.3® 
noise equivalent temperature of the thermal band on the SL-3 mission. 

To this extent then, the data quality of the thermal channel did 
influence the order of channel selection. 
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